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MOVING TOWARDS QUALITY PROGRAMS: 
Defining Criteria for Quality Program Design 
Based on Lessons from Research and Experience 

Susan P. Curnan 
Alan Melchior 
Center for Human Resources 
Brandeis University 

During the past 10 years, the employment and training community has witnessed 
a steady evolution of youth employment and education policy at the national level. 
Prompted by an ongoing stream of research and national commission reports on the 
increasingly challenging labor market and the poor basic skills of American young 
people/there has been a significant shift in national policy towards strategies that 
promote the establishment of ambitious national standards, the targeting of services to 
those with the poorest skills, and the development of more comprehensive service 
strategies, particularly those that provide a stronger link between work and education. 
These changes are evident in a number of key pieces of legislation, including the 1992 
JTPA Amendments, the School-to- Work Opportunities Act, the Goals 2000: Educate 
America Act, and the National and Community Services Trust Act. They also inform 
changes in JTPA's performance standards system as well as the development of a 
number of major national youth initiatives such as Youth Fair Chance, Youthbuild, and 
Americorps. 

At the core of this change in national policy r> the recognition that there needs to 
be a fundamental shift in the way we prepare young people, particularly economically 
disadvantaged youth, for productive citizenship and employment. Implicit in that policy 
change is a shift from an overv^helming emphasis on employment as the primar>' goal of 
youth-oriented programs to an emphasis on employahility development and a 
corresponding shift towards longer-term, more comprehensive strategies and towards the 
targeting of services to those most in need of assistance. 

While there has been substantial progress on the legislative and policy front, most 
practitioners and policy makers would agree that the translation of policy into working, 
high quality programs in the field remains a major challenge. Under JTPA. the 
investment in staff development has been minimal, and much of that has focused on 
compliance rather than program redesign or improved services.' Guidance for local 
policy makers and program managers has also been limited. Changes in performance 
standards have provided general goals, but offer little in the way of clear standards of 
quality for program design and development. 



' Berkeley Planning A-ssociatcs and Macro Systems, Inc., JTPA Staffing and Staff Training at the State 
and SDA LeveLs, Final Report (August .^0, 1990). 
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If change is to come in the ways that services are designed and dehvered to youth, 
national leadership at the Departments of Labor, Education, and others, and local 
practitioners and policy makers need to work together to define the elements of program 
quality and to make the investments in staff development needed to put those criteria or 
standards into operation. 

The purpose of this paper is to provide a starting point for this challenging task 
by suggesting a set of criteria, based on our reading of the research literature and our 
experience in the field, that we believe represent key indicators of effective employment 
and training programs for youth. It is important to note that our emphasis here is not 
on defining a new set of outcome measures, though there is clearly a need to continue to 
refine the performance standards we apply to youth employment and training programs. 
Instead, our focus is on the questions: "What are the elements of an effective program 
for at-risk youth?" "What do we know about 'what works' for at-risk youth, and how can 
we begin to translate that knowledge into standards for high quality youth programs?" 

In the discussion that follows, we argue that the answers to these questions can be 
framed in terms of four broad themes: 

1. the need to focus on youth as youth, and to address the developmental 
needs of youth within our program strategies; 

2. connecting work and learning, by creating learning-rich work experiences 
and transforming the way in which learning takes place in classrooms; 

3. providing opportunities for longer-term sequences of services that recognize 
employability development as a long-term investment for some youth, and 
that provide the support that many at-risk youth need to develop the 
higher level skills needed for long-term emplovTnent; 

4. promoting quality in a decentralized system, through significant investments in 
staff development and in gathering the data (through assessment and 
evaluation) needed for effective management and improvement. 

The discussion that follows draws on a number of sources. A major source is the 
research conducted by the Center for Human Resources at Brandeis University and 
Public/Private Ventures as part of the Youth Research and Technical Assistance 
(YRTA) Project, funded by the U.S. Department of Labor. Tlie Youth Research and 
Technical Assistance Project was a multi-year effort to review, distill, and disseminate 
fmdmgsfrom the past decade of research on youth and employment-related programs 
and services, as well as research in education, economics, psychology, sociology, and 
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political science.^ The four themes used to organize this paper were first articulated in 
the course of the YRTA Project and draw heavily on the project research. One of the 
lessons that we confirmed in that research was that there is a substantial body of 
research and knowledge about effective practices in other fields - in thfe literature on 
adolescent psychology, evaluation research on alternative schools, cognitive development 
research, and the like -• that can directly inform the design of employment and training 
programs for youth and substantially expand our knowledge base beyond the confines of 
our own, limited, evaluation research. 

At the same time, we also draw heavily on the practical program experience 
gained in the course of Brandeis* own work in the field and its synthesis of the 
experiences of practitioners working around the country. This "field-based research" 
often complements the more formal research by reinforcing the formal research findings 
and by helping to identify the practical implications for program design and 
implementation.^ 

Finally, our goal in this paper is to present the basis for a much more extended 
discussion. While we believe that research and experience do point towards criteria for 
effective practices, we make no claim that the criteria presented here represent the only 
way of defining those standards or a complete list. Rather, they are conceived as a 



One of the major products of the YRTA Project was a collection of research sv'nthesis papers aimed 
at addressing three broad questions: What programs, serv ices, and techniques best prepare youth for jobs 
and careers? What strategies of governance and management offer the related opportunity for effective 
delivery' of those services? Whit factors regarding youth, their environment, and the labor market must be 
^ addressed in providing those sci% ices? The project's two-volume collection of papers included 10 papers: 
"The F.ffectiveness of Federally-Fu.ided Fimployment and Training Strategies for Youth;" "Supportive 
Services for Youth;" "Structure and Sequence: Motivational Aspects of Programmatic Structure in 
Employment and Training Interventnns for Disadvantaged Youth"; "Critical Skills for Labor Market 
Success;" "School-to- Work Transition : Failings, Dilemmas, and Policy Options;" "Coordination, 
Collaboration, and Linkages;" "Performance Standards and Performance Management; Die Mission and 
Structure of National Human Resource Policy for Disadvantaged Youth;" and "Youth in the Nineties: 
Trends and Expected Patterns;" "Youth and the Labor Market in the Nineties." See U.S. Department of 
Labtir, Dilemmas in Youth Employment I'rofrrammm^: Findiiif^s from the Youth Research and Technical 
AssLstance Project (Washington. D.C.: U.S. Department of Labor, F.mplovment and Training 
Administration, 1992) 

■ Brandeis' recent Summer Beginninf^s project, a 12 site network of summer work and learning 
programs has provided a particularly rich set of insights as communities have worked to transform the 
concepts of hands-on learning and work-based learning into practical, working youth programs. A.s part of 
the YRTA Project, the Center for Human Resources has also developed a series of practitioner-oriented 
guides and training packages: on program design options for in-school and out-of-school youth; 
employability assessment; case management; and summer work-based learning. Those materials were 
based on the formal YRTA Project research plus case studies, interviews with practitioners and other 
field-based research efforts. 
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starting point and a framework that can inform and advance the discussion to come/ 



The Context- Youth Need Clear Standards and Local Variations 

The starting point for any discussion of the effectiveness of programs for youth is 
the recognition that there is no single program model that works for all youth. The 
young people we serve differ widely in terms of their age, skills levels, knowledge and 
experience, and life circumstances. While one young person may benefit from (and be 
ready for) a highly structured occupational training program, another may need a more 
basic introduction to work and workplace skills. 

When we talk about the criteria for effective programs, then, we need to do so in 
a context that recognizes the need for flexibility in the appHcation of those criteria to 
individual programs. We know, based on research on adolescent development, 
education, etc., that younger adolescents are different from older youth (and different 
from adults) and generally respond to different types of instructional strategies.^ We 
know that skill levels, in terms of basic academic and work-related skills, vary 
substantially among young people, ranging from youth who can read and write at only 
the most basic levels and who have never worked to those who are essentially ready to 
move into a demanding training or emplo>'ment environment.* We know that young 
mothers require a mix of services, including child care, medical services, flexible 
scheduling and the like, that may be different from other young adults and that their 
program participation depends, in large part, on the degree to which those service are 



It IS important to note that the question of how any set of criteria or standards is to be applied 
(nationally or locally, voluntary standards or program certification, etc.) is a wholly separate question, not 
addressed in this paper, and one on which there is Kiund to he much debate. 

See for example, Carnegie Commission, Turning; hums I'n'purinii American Yoiiih for the 21si 
Ceitiun. (Carnegie Commission iU-port. Volume 21, Number 1.^. June'. 1989): I'eter .Scales./1 roriraii of 
Young Adolescents in the mOs (Chapel Hill: Center for liarlv Adolescence. 1991): and Center for Human 
Resources, Future Options Education (Waltham, MA: Center for Human Restiurces. 1990) for discussions 
of sea-ices for younger adolescents. Much of the middle school movement in education, for example 
grows out of the recognition of the distinctive needs of early adolescents. In much the same vein, research 
such as Paul Osterman's study of youth in the labor market (Getting Started: -Ilie Youth Labor Market 
(Cambridge, MA: MIT Press, 1980)) suggest that older adolescents also operate in ways thai are distinct 
from that of adults, moving from job to job, testing out careers, etc. 

See the various studies by the liducationai lesting Scmce, such as Ir\^in Kirsch and Ann Jungblatt, 
Literacy: Profiles of America's Young Adults (Princeton, NJ; I-.ducational I esting Service, 1986) and 
Kirsch, Jungblatt, and C\impbell. Ikyond the School Doors: I he Ijteracv Needs of Job Seekers Served by the 
U.S. Department of Labor (Washington, D.C:.: U.S. Department of Labor, 1992)'. 'nie need to match' 
services by skill levels has been made in a number of Brandeis publications. See, for example. Working It 
Out: An Anthology- of State and Local Performance Management Strategies (Waltham, M.A: Center for 
Human Resources, 1989) and Practitioner s Guide to Program Options fur Out-of-School Youth (1992), 
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offered.' We also know that programs need to build on local resources and 
circumstances. The history of replication efforts in employment and training and 
elsewhere highlights the dual need for clear standards and local variations. 

The first point, then, is that effective program design begins with an awareness 
and understanding of the needs and characteristics of young people and the need to 
target program design (that is, to apply the elements of effective practice) in ways that 
address the needs of the particular population being served.* 

The second, related point is that program design also needs to recognize the skill 
demands of employers and the labor market. The goal of every effective employment 
and training programs for youth is to prepare young people for long-term employability. 
To meet that goal, practitioners and policy makers need a clear understanding of the 
skills, knowledge, and behaviors that are required for success in the labor market and 
integrate those skills as the goals of their programs. It is within these two parameters: 
the needs and characteristics of program participants and the needs and requirements of 
employers that program design takes place, and it is within that context that we can 
begin to define a more general set of criteria for quality programs. 



A Framework for Program Quality 

The Youth Research and Technical Assistance Project defined a set of four broad 
research and policy themes as ways of organizing the major lessons from program 
experience and research and defining the critical building blocks for effective vouth 
programs. While by no means the only framework, they represent an effort to present a 
growing body of research in a clear and understandable wav. 

Youth Development 

The first of these themes, and one of the strongest messages to emerge from the 
YRTA Project, is the need to integrate the ideas of youth and adolescent development 
into youth program design and to provide developmentally appropriate experiences 



Sec, for example, Denis Holit, Jane Ouinl, and James Rieeio, Tfw ( luillcni;c oj Scninf^ Iccnui'e 
Mothers: lessons from Projeci Redirection Manpower Development Research Corporation. 1988) and 

Janv Quint, Barbara Fink, Sharon Rouser, Ken- Chance: Implementinf; a Comprehensive Prof^am for 
Disadvantaged Young Mothers and Their Children (NY: MDRC. I9<J1) 

' One of the reasons (though not the onlv one) that so many employment and training programs ' 
produce poor results is that historically young people have been placed in available slots rather than in 
services matched to their needs. In that regard, it is important to note that most of the young people in 
the JTPA evaluation were enrolled in adult-oriented programs rather than in programs designed 
specifically to address the needs of at-risk youth. 

Center for Human Resources 
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(including work experience) for young people as part of every employment and training 
initiative. Most employment and training programs today have been designed for adults 
and older youth, with relatively little attention paid to the developmental needs of 
younger adolescents. But the research on adolescent development is clear that young 
people bring a different set of characteristics and developmental tasks to education and 
training than do adults. In the words of one of the YRTA reports, "youth coming to 
second-chance programs are undergoing the psychological, emotional, and social 
development that is an inherent part of the passage through adolescence." They are 
trying to establish an independent identity, learning to think in new ways, testing out new 
roles and relationships, learning about different behaviors and strategies. For young 
people to make the successful transition to adulthood and employability, they need an 
opportunity to practice those new skills and to master these developmental tasks.' 

There are three related sets of implications and basic criteria that flow from this 

theme; 

1. Program designs need to be age and stage appropriate. In broad terms, programs for 
younger participants need to be more exploratory in nature, have more variety in their 
activities, and include more group than individual work. Materials and activities for 
younger participants (particularly middle school programs) should not simply be 
"dumbed-down" versions of those used with older youth or adults.'" 

2. All youth employment programs need to be developmentally-oriented and include a 
range of opportunities for young people to engage in developmentally appropriate activities. 
Perhaps the most critical of these is an opportunity to participate in a task-based 
relation.ship with an adult. It is through the development of ongoing relationships adults 
- workplace supervisors, mentors, case managers, or others - that young people have a 
chance to test out new roles, practice relating to adults in work-related settings, learn 
about adult responsibilities and expectations. anJ test out their skills. But other 
elements of program design are developmentally important as well. Young people also 
need opportunities to develop positive peer relationships, to demonstrate competency 
(and hence build self-esteem, and to gain experience in dealing with a variety of work 
and social situations. Finally, young people also need an opportunity to review and 
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' See ITiomas J. Smith and Michelle Alberii Gamb<.)nc, "IZffeclivcness of Federally Funded 
Fimploymcnl Training Strategies for Youth." in Dilemmas m Youth Emplovmeni Programming: Findings 
from^ the Youth Research and Technical Assistance Project (Washington. D.C.: U.S. Department of Labor. 
1992). AJ.sti sec Michelle Gambone. Strengthening Programs far Youth: Promoting Adolescent Development 
in theJTPA System (Philadelphia; PublicTrivate Ventures. 199.^) a further study of the integration of 
adolescent development tasks in youth employment and training programs conducted for the YRTA 
project. 
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See Center for Human Resources. Future Options Education for a discussion of age appropriate 
-■er-related activities for middle-school aged youth. 
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reflect on what they are experiencing and learning so that the lessons of their own 
experience have a chance to be integrated into daily thinking and behavior." 

Not surprisingly, well-structured work experiences offer many of these 
opportunities: task-based relationships with adults, peer relationships, opportunities to 
learn and exercise skills and achiev'"ment and to function in a variety of settings and 
situations. This is a critical point, because in this context, work experience becomes a tool 
for youth and employability development and an integral pan of the adolescent development 
process rather than a final outcome. One implication of this is the need to build work 
experience back into youth program design as an integral element. A second is a new 
emphasis on the quality of work experience (in terms of the quality of the supervision 
and the skills required) as a key program effectiveness criteria.'^ 



YOUTH DEVELOPMENT 

Serving Youth as Youth 

□ Age and Stage Appropriate Program Designs 

□ Program Designs that are Developmcnially Oriented and Include Opportunities 
to Engage in Developmenially Appropriate Activities 

* Task-based AdultA'outh Relationships 

* Peer Interaction 

* Opportunities to Demonstrate Competency 

* Reflection 

* Quality Work Experience 

A Variety of Experiences (Recreation. Service. Education, Work, etc.) 

□ An Assessment Process that Can Identify the Needs and Developmental Stages 
of Young Participants and Link Them to Appropriate Senices 



" The role of reflection as part of the developmental and educational process has been particularly 
highlighted in the literature on community service learning. See for example, the various essays on 
community service in Jane C. Kendal! and Associates, Coinbinini; Sen ice c.^d Leanuitf;: A Resource Book 
for Community and Public Senice (Raleigh. NC: National StK'ietv for Internships and I"xpcricntial 
Education, 1990). 

'■ It is worth noting that the research on the impact of work experience on school-aged youth is 
increasingly pointing to the quality of the worksite, in terms of the quality of the supervision and the skills 
required by the job, as the critical factor in determining negative or positive impact. See, for example, 
David Stern. "Quality of Studcnt.s' Work Experience and Orientation Toward Work," Youth and Society, 22 
(2): 263-282. These and related studies are summarized in the YRTA Project paper by Smith and 
Gambonc, "The nffcctivcncssof Federally Funded Employment Training Strategies for Youth." 
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It is also important to recognize, however, that other experiences can supplement 
work in providing developmental opportunities. Much of the emphasis of the youth 
development movement is on the importance of providing positive, structured 
experiences for young people, including community service and recreation. Again, the 
point here is to create opportunities for young people to develop the broader social, 
emotional, and cognitive experiences that they need and will draw on in the 
workplace.'^ 

3. A youth-centered approach also requires an assessment process that can identify the 
needs and developmental stages of young participants and link them to appropriate services. 
As will become evident later in this paper, a high quality assessment system is one of the 
program effectiveness criteria that relates to all aspects of program design and 
management. 



Connecting Work and Learning 



The second major theme is the importance of strengthening the link between 
work and learning and of providing opportunities to develop basic and cognitive skills in 
a "real world" context. This theme lies at the heart of much of the transformation of 
employment and training policy during the last decade, dating back to the 1986 JTPA 
Amendments which required, for the first time, the integration of basic skills instruction 
in the Summer Youth Employment and Training Program. The benefits of combining 
basic skills education and occupational training has also been one of the long-term 
lessons from the Job Corps and the YEDPA demonstrations and directly informed 
projects such as the Jobstart demonstration.'^ 

More recently, educational research oil cognitive development and the myriad 
reports on the growing demand for applied basic and higher order thinking skills in the 



'I"hc Youth Fair Chance model rccopni/cs this particular point in its emphasis on the development 
of recreational programs as well as the core school-to-work and community-based training programs. 

The initial goal of adding academic enrichment to the summer jobs program was to reduce summer 
learnmg loss. One of the major positive findings of the STIZP program was that summer learning loss did 
take place in the absence of any inter^'ention and that programs like S'lTP, which combined a half-day of 
education with a half-day of work, could effeclivcly reduce learning loss among disadvantaged teens. See 
Jean Baldwin Grossman and Cynthia Sipe. Summer Tramitifi and Education Pro-am: Report on Long-Term 
Impacts, (Philadelphia: l>ublic,/I>rivate Ventures, 1992) and (}ary Walker and F-rances Vilella-Velez, 
Anatomy of a Demonstration: Bte Summer Traimnf; and Education Progrum (STEP) from Pilot Through 
Replication and Post-program Impacts (Philadelphia: Public/l>rivate Ventures, 1992) For the Job Corps and 
YI:DPA findings, see Andrew Hahn and Robert Lerman, mat Works in Youth Emptovmeni Policy? 
(Washington, D.C.; National Planning Association, 1985). Also see Pa'.ricia Auspos, Launching 
JOBS I ART: A Demonstration for Dropouts in the JTPA System (NY: MDRC, 1987). 
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"high performance workplace" have led to a new sof histication and understanding of the 
work and learning connection. Whereas the lesson from YEDPA and the Job Corps was 
"combine work and learning," the lessons from SCANS, America's Choice, and similar 
studies is "integrate work and learning by teaching basic skills in woik-related settings." 
In effect we have learned that simply combining basic skills education and work in the 
same program is not enough. If young people are to learn to use their skills in the 
workplace of the future, they need to learn and practice those skills in a work-related 
setting}^ 

There are a number of implications of an integrated work and learning strategy 
for program design and management: 

1. Both worksites and classrooms need to provide opportunities for active, hands-on 
learning using work-related materials and situations. In the Summer Beginnings program, 
practitioners spoke of the need to create "learning-rich work" by making the worksite a 
learning laboratory through the use of work-based curriculum and instruction. In the 
same vein, project participants also spoke of "transforming classrooms" by organizing the 
classroom into high performance, task-based work organizations and by using real work 
situations and projects as the basis for in-school learning. 

Integral to this transformation are a number of fundamental changes in the way 
in which learning, teaching, and supervision are organized. At the core of the 
transformed classroom and worksite is the idea of active learning, where youth are 
actively engaged in individual and team-oriented projects where they research, plan, 
implement, and evaluate the work. In that context, teachers become guide, coaches and 
facilitators rather than lecturers, and need to bring an understanding of workplace skills 
and how to create situations where young people can learn and use those skills. 
Worksite supervisors also take on a somewhat different role, with a commitment to 
learning and an understanding of how to create learning opportunities on the job. 
Finally, worksite supervisors and teachers need to be able to work together so that 
classroom-based and worksite-based learning reinforce each other. 

2. Programs also need to develop a broad set of workplace-related basic skills that meet 
the needs and expectations of employers. One of the major elements of the work and 
learning connection is the need to interpret "basic skills" in the broadest possible context 
(the SCANS term is "workplace know-how") and to teach not only the foundation skills 



" Sec U.S. Department of I.afxir, Secrciar\'s Commis.sion on Achieving Necessary Skills, »7iu; Work 
Requires of Schools. A SCANS Report for America 2000 (Washington, D.C.: U.S. Department of Labor, 
Wl) and Commission on the Skills of the /\mcrican Workforce, /Imer/c^^ i- Choice: Hif-h Skills or Low 
Wages! (Rochester, NY: National Center on Education and the IZconomy, 1990). One of the more 
influential of the educational researchers on this issue is Lauren Resnick. See her paper, "Learning In 
School and Out," Educational Researcher 16: \?,-2Q (1987) for her argument on the need to teach in "real 
world" settings. 
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(reading, writing, calculation), but the social and interpersonal skills needed to function 
in a flexible, interactive workplace. To accomplish that goal, once again, programs need 
to provide opportunities for participants to practice a wide variety of skills -- teamwork, 
communication, problem-solving, etc. -- in a practical setting. Here, too, one might note, 
a well-structured and supervised workplace provides at least one setting in which skills 
can be learned and practiced in context. 

3. Ongoing assessment needs to be an integral part of program design, and the assessment 
approach needs to match the skills being measured. Few of the skills and competencies 
included in "workplace know-how" can be measured through traditional paper and 
pencil, multiple choice tests. In order to assess workplace-related skills effectively, 
programs need to adopt a mix of performance-based assessment strategies. 

CONNECTING WORK AND LEARNING 

Sinicturing Active Learning Environments 
for Education and Employability Development 

□ Opportunities for Active, Hands-On Learning Using Work-Related Materials and 
Situations (Transforming Classrooms and Creating Learning Rich Worksites) 

Youth Actively Engaged in Researching, Planning, Implementing, and 
Evaluating Individual and/or Team-oriented Projects 

* Teachers as Guides, Coaches and Facilitators 

* Teachers Understand Workplace Know-How/Basic Skills and Can 
Create Curriculum Where Young People Can Learn/Use Those Skills 
Worksites Committed to a Balance of Work and Learning 
Worksite Superx'isors Understand Workplace Know-How and Have 
Ability to Create "Learning Rich" Work for Youth 

□ A Broad Set of Workplace- Related Basic Skills that Meet the Needs of 
Employers 

* A Combination of Basic "Foundation" Skills and Workplace 
Competencies (SCANS Skills as an Initial Framework) 
Employer and Community Involvement in Defining Skills 

□ Ongoing. Performance-based Assessment 

□ A Clear Connection from the Educational Process to the Labor Market 

A Career Plan and Path to Employment or Further Education and 
Appropriate Connecting Activities 
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4. Finally, the research and practice associated with school-to-work transition also 
makes clear that programs need a clear connection from the educational process into the 
broader labor market - a career plan and path to education or employment that reduce the 
"milling around" that is common to young people in the labor market and that helps 
young people ensure that they are gaining the skills they need. In the language of the 
new School-to- Work Opportunities Act, these "connecting services," which include career 
counseling, job search assistance, and the like represent the link between an integrated 
work and learning strategy and its ultimate outcome of a job. 



Extending Services Over Time 

The third basic theme is the need to develop a sequence of services that extends 
over time as well as a comprehensive mix, of services. Experience, common sense, and 
research point in the same direction: the more at risk the population, the more extensive 
services must be to achieve substantial employment and income impacts. In the same 
vein, to the extent that we view employability development as a developmental process 
rather than a one-time inien'ention, we need to provide the capacity for young people to 
move through a series of programs and learning experiences over an often extended 
period of time. 

Accomplishing these goals, however, requires a serious rethinking about both program 
design and our expectations about how young people gain the skills and experience needed to 
be employable. In many ways, our existing emplovTnent and training system is 
constructed on the assumption that young people (and adults) ente7 a' program and stav 
there until they are job-ready. Within that paradigm, employability development is 
viewed as a one-time intervention, and longer-term services simply means staving within 
a single program for a longer period of time. 

But research and our own experiences in ilie labor market point to the fact that 
for many youth, education and the acquisition of career-related skills and experience 
takes place in stages, and often in fits and starts. Research on both high school 
dropouts and college students tells us that many young people drop out and back into 
the educational process over a period of time. Reports from the Jobstart demonstration. 
New Chance, and other programs for youth also tell us that young people are regularly 
forced by external factors to interrupt their education and training - to care for children, 
earn a living, or deal with a family crisis - even when a full set of "comprehensive" 
supports are available. Research on the entry of young people into the labor market 
also argues that initially youth often move from job to job (and possibly program to 
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program) until they find a "match" that fits their needs and lets them move forward.*' 



The point here is a simple one: as we think about longer-term, comprehensive 
services for young people, we need to recognize that long-term employability 
development is often made up of a series of short-term commitments. At the program 
and "system" level, we need to design strategies that allow for this "in and out" process 
and that provide opportunities to sequence education and training in flexible manner 
over a period of time. Some of the implications for program design include: 

1. Individual programs and community-level "systems" need to provide (or flexible 
entry and exit through a developmental sequence of programs and services. The idea of open 
entry/open exit within programs is a familiar one, and has long provided a means for 
youth and adults to match program participation with the other demands on their time. 
What is needed in addition is a mechanism for open entry and exit within a system of 
programs, so that young people can complete a set of services, leave to work, and return 
to upgrade to the next skill level over a period of time, without having to terminate and 
re-enroll or lose his or her eligibility for services. An essential part of the process is the 
provision of post-program follow-up services that provide an ongoing point of contact for 
participants as they move back and forth between programs or between training and 
work.'^ 

2. Programs and communities also need the capacity to match youth to appropriate 
services and to track progress over time. The basic element here is a community-wide 
assessment and case management system that has the capacity to identify interests and 
needs on an ongoing basis, develop clearly defined goals and plans, help young people 
move between programs, provide longer-term follow-up services (see above) and track 
progress over time. To accomplish this, youth-serving programs and institutions need to 
establish mechanisms for sharing assessment data, common protocols for accessing 



The 1991 Department of Education report on Dropout Rates in the United States (Washington, D.C.: 
U.S. Department of Education, National Center on Education Statistics, NCES 92-129, 1992) notes that 
nearly all the dropouts in a 1988 survey reported that they planned to complete their high school 
education in the future. An earlier report {Dropout Rates m the United States, I'JSS) notes that 
approximately 46<7r of the dropouts in a 1980 cohort of students had earned a diploma or acquired a GEO 
by 1986. and an additional 12^; were in the process at that time. For college retention, see Vincent Tinto, 
Leaving College: Rethinking the Causes and Cures of Student Attrition (Chicago: University of Chicago Press, 
1987). Tir.to notes, for example, that only A()7r of those completing college do so in 4 vears. and that 
approximately aO'/f of those who leave college are "stopouts" who later return, sometimes after significant 
periods of time. 

A recently completed study by Andrew Hahn at Rrandeis of the Quantum Opportunities Program 
(OOP), funded by the Ford Foundation, confirms the importance of this ongoing contact over an extended 
period. That program provided a mix of basic skills, career planning, support and case management 
services over a four year period to high risk youth in four sites. Even in those sites where services were 
relatively limited by the fourth year, the extended contact proved to have an impact on participants. 
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services, and a shared referral process. 



3. Programs and communities need to establish community-wide interagency 
partnerships to provide for both long-term and comprehensive services. Long-term and 
comprehensive services require a community-wide strategy to ensure the availability of 
services, to provide for easy transition from program to program (or from summer to 
school year, for example), and to provide a broad mix of funding to support a flexible 
mix of services.'' 

4. Finally, the ideal system will also provide a variety ofp ogram strategies for 



EXTENDING SERVICES OVER TIME 

Providing a Sequence as well as a Mix of Services 

□ Flexible Entry and Exit Through a Developmental Sequence of Programs and 
Services 

* Open Entry/Exit within Programs 

* Open Entry/Exit within a System of Programs that Allows Participants 
to Move Through a Succession of Activities/Experiences 

* Provision an Ongoing Point of Contact and Long-Term Post-Program 
Services 

□ The Capacity to Match Youth to Appropriate Scnices and Track Progress Over 
Time 

Community-Wide Assessment and Case Management System 
including Mechanisms for Sharing Assessment Data. Common 
Protocols for Accessing Services, and a Shared Referral Process 

□ Community-Wide Interagency Partnerships to Provide for Long-Term and 
Comprehensive Services 

□ A Varie'y of Program SiratcgieslOpiions for Pariicipanis 



" ITic Youth Fair Chance initiative represents a move in this direction. The Department has called 
for YI-C communities to establish a "community management system" in recognition of the need for a 
systems approach to delivering a range of services. 

ITie Youth Fair Chance design also begins to address this issue through the requirement of a 
Community Rcsourcc/Advisorv- Board and planning for comprehensive seivices for ail vouth in a saturation 
strategy. 
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participants to choose from. One of the clear lessons from experience is that no single 
program strategy works for all youth. Not only do programs need to address different 
age and skill levels among participants, but ideally participants should have an 
opportunity to select a program that matches their learning style. We know, for 
example, that 25% of the participants in conservation and service corps programs drop 
out within the first month, presumably because they realize that the corps approach does 
not work for them.^ A "youth-centered" or "customer-driven" approach suggests that 
the more options a community can offer young people for acquiring employability skills, 
the more likely a young person will find a good "match" and gain the skills needed for 
employment. 



Promoting Quality in a Decentralized System 

As a largely decentralized system both locally (through networks of service 
providers) and nationally, the employment and training system faces a constant challenge 
in promoting the quality and effectiveness of its programs. As the emphasis on 
providing services to more "at-risk" youth grows, and as the employment and training 
system moves towards the provision of more complex, comprehensive, and longer-term 
ser\ices, the issues of program qualit>' and effectiveness are becoming more critical. One 
of the implications of these developments (and one of the major lessons of initiatives 
such as Summer Beginnings) is the need for significant and strategic investment in 
capacity building among employment and training professionals. We need to recognize 
that in the same way that American competitiveness depends on the development of a 
highly skilled workforce, the qualitN- of employment and training services also ultimately 
depends on building the skills of the professionals responsible for implementing them. 

While capacity building is key. the implementation of effective strategies for 
defining and measuring program quality is also vital. For the employment and training 
system, this means an investment in ongoing assessment and evaluation and a system for 
performance management based on clearly defined goals and outcomes. In that context, 
three key elements stand out in promoting quality and effectiveness at the local level 
include: 



1 Active investment in capacity building and professional development. Professional 
development needs to move beyond occasional conference attendance and training on 
compliance issues. Effective capacit>' building needs to take place on a regular basis an^ 
to address such ser\ice-related issues as adolescent development, workplace skills and 
the changmg labor market, curriculum and instruction for work-based learning, and 
performance-based assessment to ensure that all staff have the skills and knowledge 



Sec Alvia Branch. Sally I.iedcrman, and 'Chomas J. Smith. Youih Consen aiton and Sen ice Corps: 
Imdingsfrom a National Evaluation (Philadelphia; PublicTrivatc Ventures, 1987). 
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needed to put effective programs into operation.'" 

2. An assessment system that can provide accurate information for matching youth to 
services, assessing gains, and tracking progress through a sequence of services. As suggested 
in earlier sections, an effective assessment system is individualized and ongoing, 
performance-based, and functionally-oriented. Assessment information on the skills of 
participants, skill gains, the degree to which program services match the needs of 
participants, and relative program performance provides a powerful tool for program 
management and continuous improvement. 



PROMOTING QUALITY IN A DECENTRALIZED SYSTEM 

Assuring Quality Outcomes for Youth, Employers, 
and the Community 

□ Active Investment in Capacity Building and Staff Development 

* Ongoing Staff Development 

* Program Design/Service Oriented 

□ An Assessment System that Provides Accurate Information for Matching Youth 
to Senices, Assessing Gains, and Tracking Progress through a Sequence of 
Services 

* Individualized and Ongoing 

* Performance-based 

Used as Tool for Program Management 

□ A Performance Management System Based on Clearly Defined Program 
Outcomes 

Clearly Defined. Meaningful Goals and Participant Outcomes, 
Developed with Employers and the Community 
Strategies for Evaluating Program Performance and Customer 
Satisfaction (Assessment and Evaluation) 

* Clear Accountability 

A Process for Reporting Results to the Broader Community 



The "subject specific" training sponsored by the Department of l.aKir represents an important start. 
I lowever, this training needs to be expanded, better targeted, and replicated on the state and local level to 
have an impact on overall program quality. 

Center far Human Resources, 
Brandeis Unh ersity- 

•f ^ 
1 ^ 



July 1994 
15 



3. A clearly defined, meaningful set of program outcomes that form the basis for an 
ongoing performance management system. An effective performance management system 
might include a clearly defined set of goals and participant outcomes, developed with 
employers and the community (perhaps beginning with the SCANS skills as a 
framework); strategies for evaluating both program performance (through assessment) 
and "customer" satisfaction; a means for establishing accountability for performance, and 
a process for reporting results to the broader community.^ 

Summary 

The elements outlined here are by no means comprehensive. But they represent 
a broad framework that, we believe, can stand as a starting point for a more substantial 
discussion of program quality and effectiveness. As national policy continues to move 
towards a more youth and development-focused vision of employability development, we 
need to find new ways of translating that vision into the day-to-day operations of local 
programs and practitioners. Clearly one step in that process is the translation of 
research and experience into principles and criteria that can be used by local policy 
makers and professionals to guide program design and to establish standards that 
promote high qualit)' services for young people. 



The- process recently used by the city of Little Rock, with assistance from the (\-nter for Human 
Resources, stands as a useful model for this process. As part of developing a eiimmunity-wide 
employability strategy. Little Rock New Lutures and the Little Riiek Youth LmpK)yment i'oliev Committe 
(a group which includes the PIC, schools, empKiyers and iithers) engaged employers, educators, youth, 
community-based organi/.-itions, and others in locus groups to define and build ciinsensus on a set of ' 
appropriate employability outci)mes. The community is ni)w in the priKcss of orienting teachers, worksite 
super\'isors, counselors, and funders lo broaden the consensus and to implement the outcomes as part of 
progra-n planning and management. 
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